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2.  Objectives  (unchanged) 

Students  will  participate  in  an  interdisciplinary  research  project  on  oculomotor 
localization.  The  main  objective  of  the  project  is  to  understand  how  human  beings  direct  saccades 
accurately  to  spatially*extended  targets.  The  projects  to  be  undertaken  are  based  on  work  from 
die  FI's  oculomotor  laboratory,  demonstrating  a  spatial-pooling  process,  which  automatically  and 
effortlessly  uses  information  in  a  selected  eccentric  target  form  to  compute  a  single,  centrally- 
located  saccadic  landing  position.  Now,  we  want  to  understand  how  this  spatial-pooling  process 
works  and  to  develop  an  information-processing  model  that  accurately  predicts  saccadic  landing 
positions.  This  project  will  be  valuable  both  for  understanding  how  the  brain  represents  the 
location  of  spatially-extended  targets  and  for  developing  efficient  algorithms  that  direct  the 
sensors  or  arms  of  robots  to  specific  locations  within  spatially-extended  targets. 

Our  approach  is  novel  in  its  interdisciplinary  nature:  it  is  an  attempt  to  extend 
nuthematical  models,  developed  to  account  for  human  visual  detection  and  discrimination,  to  the 
"higher-level"  problem  of  motor  localization.  The  project  involves  a  collaboration  between  the 
PI,  whose  area  of  expertise  is  human  oculomotor  control,  and  scientists  at  the  nearby  Samoff 
Laboratories,  who  have  been  developing  mathematical  models  of  visual  discrimination.  The 
AASERT  students  participate  in  boA  the  empirical  and  theoretical  phases  of  the  collaboration, 
while  pursuing  a  course  of  study  that  emphasizes  training  in  cognitive  science,  visual  science, 
psychophysics,  mathematical  modeling,  and  motor  control 

3.  Status  of  effort 

Two  graduate  students  have  been  supported:  Dan  Bahcall  and  James  McGowan. 

Bahcall's  work  has  concentrated  on  higher-level  aspects  of  oculomotor  control,  namely,  spatial 
attention.  McGowan  has  been  studying  the  lower-level  sensory  processes  that  guide  saccades  to  a 
central-reference  position  within  attended  targets.  Both  students  have  completed  courscwork  in 
sensory  processes,  perception,  computer  vision,  computadonal  models  of  cognition,  and  attention, 
as  well  as  graduate  level  courses  in  advanced  stadstical  theory,  neural  networks,  and  mathemadcal 
logic.  Training  has  been  broadened  through  interaction  with  specialists  in  vision  research  and 
mathemadcal  psychology:  (1)  J.  Bergen  (Samoff  Labs),  who  is  collaborating  with  McGowan  on 
saccadic  localization  experiments  using  spatially-filtered  targets,  (2)  C  Chubb  (Rutgers),  who  has 
collaborated  with  McGowan  on  saccadic  localization  experiments  and  also  worked  independently 
with  McGowan  on  models  of  visual  motion  detection,  (3)  B.  Dosher  (U.C.  Irvine)  who  is 
collaborating  with  Bahcall  on  experiments  in  spatial  attention.  Two  undergraduate  students  have 
been  supported:  E.  Anderson,  who  collaborated  with  Dosher  and  myself  on  experiments  on 
saccades  and  attention  and  C.  Araujo,  a  computer  science  major,  and  now  a  graduate  student  in 
philosophy,  who  has  begun  working  on  experiments  looking  at  the  cognitive  control  of  eye 
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movements  during  visual  search.  In  addition,  D.  Melcher,  a  graduate  student,  has  been  working 
on  saccadic  localization  of  ^atially-structuied  targets. 

4.  Accomplishments  and  new  findings 

fal  Saccades  and  attention:  AASERT  student:  E.  Anderson  (undergraduate  honors 
thesis).  These  experiments  explored  the  nature  of  the  selective  filter  that  guides  saccades  to 
chosen  objects.  We  asked  whether  the  a  single  selective  filter  serves  both  perception  and  eye 
movements  or  whether  the  saccadic  system  employed  its  own,  independent  filter.  If  a  shared 
selective  filter  is  involved,  then  it  would  not  be  possible  to  prepare  to  look  at  one  object  while  at 
the  same  time  paying  full  perceptual  attention  to  something  else.  We  devised  several  "dual-task" 
experiments  (concurrent  measures  of  saccades  and  perception)  requiring  subjects  to  look  at  one 
target  while  identifying  another.  Considerable  effort  was  devoted  to  development  of  dual-task 
methods  that  placed  subjects'  strategies  under  tight  experimental  control.  We  found  that 
identification  of  a  perceptual  target  (a  randomly-chosen  letter)  was  better  at  the  saccadic  goal 
than  elsewhere.  Surprisingly,  it  was  possible  to  accurately  identify  a  letter  at  a  location  other  than 
the  saccadic  goal  by  increasing  saccadic  latency  only  10-30%.  These  results  show  that  saccadic 
programming  does  make  demands  on  perceptual  attention,  but  the  atttentional  demands  are 
modest  To  explain  the  results,  we  proposed  that  attention  is  not  involved  in  saccadic 
programming  until  shortly  before  saccadic  execution,  and  that  it  is  at  this  time  that  the  spatial 
parameters  of  the  saccade  are  determined.  One  way  to  implement  this  process  would  by  involving 
two  systems,  one  that  determines  the  saccadic  goal  (controlled  by  means  of  selective  attention) 
and  another  that  triggers  the  saccade.  By  setting  the  saccadic  trigger  to  launch  the  saccade  in 
response  to  die  attentional  shift,  scanning  would  be  botii  rapid  and  accurate  while  at  the  same 
time  minimizing  the  need  for  time-consuming  "on-line"  decisions  about  when  and  where  to  aim 
the  eye.  (Kowler,  Anderson,  Dosher,  and  Blaser,  1995). 

fb)  Attentional  interference:  AASERT  student:  D.  Bahcall.  We  performed  a  perceptual 
version  of  the  experiment  described  above.  Instead  of  lopking  at  one  target  and  trying  to  identify 
another,  we  asked  subjects  to  identify  two  targets  in  a  circular  display  (rad  4  deg)  of  24  letters. 
Contrary  to  expectations,  we  found  that  target  identification  improved  with  increasing  target 
separation.  The  interference  was  attentional  and  not  sensory  because  the  display  configuration 
did  not  vary  with  target  separation.  These  novel  results  demonstrate  spatially-local  limits  on 
attentional  resources.  Attentional  interference  is  useful  in  that  it  enhances  the  perceptibility  of  a 
single  target  against  nearby  neighbors.  It  also  suggests  that  optimal  visual  performance  would  be 
achieved  in  displays  in  which  important  features  are  widely-spaced.  Bahcall  has  modeled  the  data 
using  difference-of-Gaussian  filters.  (Bahcall  and  Kowler,  1995  and  submitted  ms). 

(c)  Saccades  to  dot  clusters:  (MpGowan,  Kowler,  Shaima  and  Chubb,  in  press): 

AASERT  student:  J.  McGowan^  The  pooling  process  that  guides  saccades  to  spatially-extended 
targets  was  investigated  by  studying  saccades  made  to  4  deg  diam  clusters  of  20  random  dots. 
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Landing  positions  came  close  to  the  center.of.gravity  of  the  patterns.  Precision  was  quite  good, 
with  SDs  around  the  center-of-gravity  equal  <  10%  of  stimulus  eccentricity.  Correlations  of 
saccadic  landing  position  with  the  presence  of  a  dot  at  any  display  location  showed  that  dots  from 
the  entire  display  were  taken  into  account  in  determining  the  landing  position  of  the  saccade.  The 
prediction  of  saccadic  landing  position  was  improved  by  reducing  the  weight  assigned  to  a  dot 
due  to  the  presence  of  neighboring  dots,  i.e.,  dots  in  sparse  clusters  were  more  influential  than 
dots  in  dense  clusters.  The  results  can  be  accounted  for  by  a  2-stagc  localization  model.  The  first 
stage  consists  of  an  airay  of  detectors  centered  around  different  portions  of  the  pattern  and  the 
second  stage  averages  the  'local  signs’  of  the  detectors. 

fdl  Saccades  to  spariallv-filtered  targets:  fin  prognessV  AASERT  student;  J.  McGowan. 
The  spatial  pooling  process  is  being  investigated  by  studying  the  precision  of  saccades  directed  to 
Gabor  patches  (sinusoidal  gratings  whose  contrast  is  modulated  by  a  Gaussian  envelope).  The 
main  objective  is  to  find  out  whether  saccades  use  the  same  set  of  spatial  filters  as  the  perceptual 
system,  or  whether  some  spatial  frequencies,  although  visibk,  constitute  ineffective  (i.e.,  noisy) 
targets  for  saccades.  Saccadic  localization  remains  precise  over  a  wide  range  of  spatial 
frequencies  (.5  to  10  cpd)  and  envelope  sizes  (SD=.5  to  1  deg),  with  precision  suffering  only 
when  contrast  reaches  values  close  to  threshold.  These  results  show  that  saccadic  localization  is 
not  dependent  on  particular  banks  of  first-stage  visual  filters,  (Kowler,  McGowan,  Bergen  and 
Lubin.) 


(e)  Saccades.  taspatiallv-extended  targets:  shapes  and  surfaces  va.  elements.  (AASERT 
Student  D.  Melcher).  The  targets  for  saccades  were  composed  of  dots  arranged  so  that  diey 
looked  like  coherent  shapes  with  obvious  boundaries,  rather  than  clusters  of  random  dots. 
Saccades  landed  at  the  center-of-area  of  the  shape,  not  at  the  centroid  of  the  dot  locations. 
Landing  positions  were  unaffected  by  changing  the  density  of  dots  along  the  contour  or  by  addling 
dots  to  the  interior  of  the  form.  Dense  clusters  of  unattended  dots  superimposed  at  an  arbitrarily 
chosen  locations  either  on  the  boundary  or  inside  the  shape  had  no  effect,  Saccades  to  dot 
clusters  created  by  randomly  perturbing  the  locations  of  interior  or  contour  dots  landed  closer  to 
the  center  of  area  of  the  nominal  shape  than  to  the  centroid  of  dot  locations.  Figures  1-3  show 
sample  stimuli  and  the  obtained  mean  landing  positions.  (Melcher  and  Kowler,  ms  in  draft  fonn). 

The  results  show  that  the  saccadic  system  represents  the  locadon  of  the  object  irrespective 
of  its  component  elements  or  internal  structure.  This  agrees  with  intuitions  that  location  is  a 
fundamental  property  of  an  object  and  should  be  independent  of  how  an  object  is  constructed. 

The  contribution  of  these  experiments  was  to  show  that  a  precise  representation  of  object 
location  is  available,  that  such  a  representation  is  actually  used  to  guide  saccades  (in  contrast  to 
an  element-based  representation),  and  tiiat  the  representation  is  accessible  with  no  special  effort 
required  to  either  find  the  landing  position  or  to  avoid  an  influence  of  element  spacing  or  internal 
structure. 

It  is  possible  that  the  object  representations  that  guide  saccades  can  be  generated  by  the 
local  sign  model  described  above  (see  c,  above)  if  we  allow  the  responses  of  local  units  to  be 
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sensitive  to  spsitial  position  of  the  elements  falling  in  their  receptive  ^elds.  So,  for  example,  3 
collinear  elements  would  produce  a  stronger  response  than  3  haphazardly  positioned  elements. 
Follow-up  experiments  will  use  sparser  clusters  of  dots  (n<10)  to  test  this  idea.  We  will  be 
looking  for  the  pooling  algorithm  that  gives  the  best  prediction  of  landing  position.  Essentially, 
we  are  looking  for  the  rules  used  to  generate  a  higher-order  representation  of  shape  from 
component  elements.  The  saccadic  localization  response  is  a  novel  way  of  testing  such  models. 

(0  Saccades  in  visual  search.:  This  is  a  new  project  in  early  pilot  stages  and  thus  must  be 
described  in  somewhat  greater  detail  than  those  above.  The  objective  is  to  apply  our 
understanding  of  saccadic  control  (studied  in  projects  a-e)  to  understand  performance  in 
naturalistic'  and  puiposeful  tasks  requiring  saccades  (Araujo,  Kowler  and  Pavel) . 

We  have  been  studying  visual  search.  There  are  two  major  constraints  on  determining  a 
search  path.  One  is  the  use  of  probabilistic  information  about  where  targets  are  likely  to  be 
located;  we  assume  that  a  good  strategy  would  be  to  search  first  in  those  locations  most  likely  to 
contain  the  target.  The  second  is  constraints  imposed  by  the  mechanisms  that  have  to  carry  out 
the  search.  If,  for  example,  two  high  probability  locations  are  far  apart,  and  there  is  a  great  cost 
involved  in  traveling,  it  might  be  better  to  search  nearby  locations  first,  even  if  this  means  visiting 
a  low  probability  location  before  a  high  probability  location.  The  main  question  is  how  human 
beings  combine  these  constraints  to  determine  the  search  path.  Any  procedure  used  to  combine 
constraints,  which  we  will  study  in  the  context  of  an  eye  movement  task,  could  presumably  be 
applicable  more  generally  to  various  kinds  of  searches. 

We  are  beginning  with  a  simple  search  task  in  order  to  identify  constraints  that  apply  to 
saccades  and  find  out  how  these  constraints  trade  off  with  locational  probability.  In  the  task 
(Figure  4)  an  observer  must  find  as  fast  as  possible  a  single  letter  T  embedded  in  either  one  of  two 
dense  arrays  of  L’s  separated  by  6  deg.  Before  the  trial  the  observer  is  told  the  probability  that 
the  T  will  appear  in  the  left  or  right  array.  Letter  sizes  and  spacing  were  chosen  so  that  in  the 
actual  displays  it  was  not  possible  to  discriminate  a  T  from  an  L  without  looking  right  at  it  The 
distance  between  initial  fixation  position  and  either  array  of  letters  is  varied  on  each  trial  (this  is 
done  by  setting  the  position  of  the  fixation  crosshair  between  the  2  arrays).  Data  shown  in  Figure 
5  are  latencies  of  the  initial  saccade  Geft  graphs)  and  intersaccadic  interval  between  first  and 
second  saccade  (right  graphs)  for  one  subject  as  a  function  of  initial  fixation  position.  The  two 
lines  in  each  graph  show  performance  when  the  high  probability  location  was  on  the  left  (solid 
line)  or  right  (dashed  line).  For  this  experiment  the  subject  kept  to  a  consistent  strategy  of  always 
looking  at  the  high, probability  location  first.  The  results  show  that  she  was  able  to  do  this  quickly 
when  the  high  probability  location  was  near  the  initial  fixation  position,  but  required  considerable 
extra  time  when  the  high  probability  location  was  far  away.  This  extra  time  was  reflected  not 
only  in  the  latency  of  the  initial  saccade  (left  graphs),  but  also  in  the  time  before  the  second 
saccade  (right  graphs),  an  interesting  outcome  because  the  size  of  the  second  saccade  was 
independent  of  the  initial  fixation  position.  This  means  that  the  results  are  due  in  part  to  the  cost 
of  using  an  awkward  strategy  (i.e.,  look  at  the  high  probability  location  first  regardless  of 
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distance)  rather  than  solely  cost  of  making  large  saccades.  Given  the  cost  of  jumping  across  the 
display,  a  better  strategy  might  have  been  to  take  distance  into  account. 

Once  some  of  the  geometric  constraints  on  saccades  are  identified  (e.g.,  looking  over 
large  distances  is  difficult)  it  will  become  possible  to  design  displays  that  pit  probabilistic  against 
geometric  constraints  and  find  out  how  observers  elect  to  trade  one  for  the  other.  These 
experiments  could  include  the  use  of  displays  which  change  over  time  in  predictable  ways  so  that 
die  observer’s  task  becomes  one  of  counting  how  many  targets  appear  across  a  sequence  of 
frames.  This  dynamic  situation  is  interesting  because  die  observer  is  required  to  use  precisely 
timed  saccadic  sequences,  which  will  introduce  yet  another  set  of  constraints  on  saccades  having 
to  do  with  die  tendency  to  pre-plan  sequences  of  movements  (Zingale  and  Kowler,  1987). 

Using  dynamic  displays  is  also  advisable  because  many  important  search  situations  require 
detecting  targets  that  will  appear  only  briefly.  In  such  difficult  situations  -  where  the  eye  must  be 
in  the  right  place  at  the  right  time  and  there  is  not  much  margin  for  error  —  it  becomes  of  great 
importance  to  achieve  both  accurate  saccadic  control  and  an  optimal  search  strategy.  The  task 
becomes  one  of  generating  a  correct  prediction,  in  space  and  time  about  target  appearance  and  of 
transmitting  these  predictions  to  the  oculomotor  system.  Our  research  has  already  shown  that  the 
sensory,  motor  and  attentional  mechanisms  are  in  place  to  support  accurate  and  precise  saccadic 
control.  The  major  limitation  on  the  effectiveness  of  eye  movements  in  real  tasks  may  well  be  the 
ability  to  form  accurate  predictions  and  translate  these  predictions  into  plans  for  movement 


Relevance  and  applications:  The  experimental  results  are  relevant  to  understanding  the 
basic  visual,  cognitive  and  motor  mechanisms  that  support  accurate  saccadic  scanning.  An 
understanding  of  both  the  attentional  requirements  of  saccades  and  the  visual  displays  that  support 
accurate  and  precise  scanning  will  contribute  to  the  design  of  visual  displays  that  produce  optimal 
behavior,  i.e.,  establishing  a  fast,  accurate  and  precise  scan  pattern  while  at  the  same  time  freeing 
the  most  attentional  resources  for  concurrent  perceptual  and  cognitive  tasks. 

Our  results  thus  far  indicate  that  with,  the  displays  we  have  used  the  oculomotor  and 
cognitive  systems  function  with  remarkable  efficiency;  For  example,  by  linking  saccadic  target 
selection  to  the  same  attentional  filter  that  serves  perception  it  becomes  unnecessary  for  observers 
to  make  a  separate  decision  about  where  to  aim  the  eye.  The  line  of  sight  will  automatically  go  to 
the  attended  region  after  a  "go"  signal  is  issued  to  trigger  the  movement.  Moreover,  the 
limitations  on  the  attentional  demands  of  saccades,  which  we  have  observed,  mean  that 
considerable  resources  are  available  for  selection  of  potential  targets  before  the  saccade  is 
initiated.  Finally,  the  finding  of  a  highly-precise  spatial  pooling  process  means  that  observers  need 
only  select  a  target  object;  the  precise  endpoint  of  the  saccade  is  determined  automatically  by 
lower-level  visuomotor  processes  that  do  not  require  deliberate  or  cognitive  intervention.  All  of 
these  principles  may  prove  of  value  in  the  design  of  artificial  visual  systems  that  must  aim  sensors 
to  selected  targets  in  large  visual  scenes. 
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It  is  worthwhile  noting  that  our  information  about  saccadic  scanning  of  natural  visual 
scenes  represents  normal  perfonnance  under  optimal  conditions,  i.e.,  high-contrast,  highly-visible 
displays  inspected  by  highly-motivated  and  attentive  observers.  The  performance  under  such 
conditions  is  remarkably  good.  These  results  serve  as  an  important  baseline  with  which  to 
evaluate  the  expected  deterioration  of  performance  when  optimal  viewing  conditions  are 
unavailable,  either  because  the  visibility  of  the  display  or  the  cognitive  state  of  the  observer  is 
impaired. 


5.  Personnel  supported 
Graduate  students; 

Dan  Bahcall  (1994-1996) 

James  McGowan  (1994-1996) 

David  Melcher  (summer  1995  and  1996) 
Undergraduate  students; 

Christian  Araujo  (1994) 

Eric  Anderson  (1994) 
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Kowler,  E.,  Anderson,  E.,  Dosher,  B.  and  Blaser,  E,  (1995)  The  role  of  attention  in  the 
programming  of  saccades.  Vision  Research,  1897-1916. 

McGowan,  J.,  Kowler,  E,,  Shaima,  A.  and  Chubb,  C.  Saccadic  localization  of  random  dot  targets. 
Vision  Research,  in  press. 

Bahcall,  D.  and  Kowler,  E.  Attentional  interference  at  close  spatial  separations.  Submitted. 

7a.  Talks 

Kowler,  E.,  Dosher,  B.,  Anderson,  E.  and  Blaser, E.  Saccadic  eye  movements  and  shifts  of  visual 
attention.  Psychonomic  Society,  November,  1993. 

Bahcall,  D.O.  and  Kowler,  E.  (1995)  Attentional  interference  at  close  spatial  separations. 
Investigative  Ophthalmology  and  Visual  Sciences  Supplement,  36,  S901. 

McGowan,  J.,  Kowler,  E.,  Sharma,  A.  and  Chubb,  C,  (1996)  Precise  saccadic  localization  of 
random  dot  targets.  Investigative  Ophthalnnology  and  Visual  Sciences  Supplement,  37,  S  524 
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Bahcall,  D.O.  and  Kowler,  E.  (1996)  Attentional  interference  at  close  spatial  separations. 
European  Conference  on  Visud  Perception. 

McGowan,  J.,  Kowler,  E.,  Shaima,  A.  and  Chubb,  C,  (1996)  Precise  saccadic  localization  of 
random  dot  targets,  European  Conference  on  Visual  Perception. 

7b.c.  None 

8.  New  discoveries,  patents,  inventions.  None 


7 


SE^T  by: RUTGERS  UNIVERSITY 


;  3-17-98  ;  3:25PM  : 


7324452263^ 


202  404  7475:#10 


Figure  1.  Mean  landing  positions  of  subject  BS  to  four  different  targets.  In  the  actual  experiment,  target 
orientation  was  varied  and  each  x  shows  mean  landing  position  for  a  different  orientation.  The  large  crosshair 
shows  the  centroid  of  the  line  connecting  the  dots  (for  the  2  targets  on  the  left)  and  the  centroid  of  the  surface  (for 
the  2  targets  on  the  right).  Landing  positions  correspond  to  these  centroids,  ignoring  differences  in  dot  spacing. 


Figure  2.  Mean  landing  positions  of  subject  BS  to  two  different  targets.  In  the  actual  experiment,  target 
orientation  was  varied  and  each  x  shows  mean  landing  position  for  a  different  orientation.  The  large  crosshair 
shows  die  centroid  of  the  surface.  Landing  positions  conespond  to  the  centroid  of  the  surface,  not  the  center  of 


Figure  3.  Mean  landing  positions  of  subject  BS  to  four  different  targets.  In  the  actual  experiment,  target 
orientation  was  varied  and  each  x  shows  mean  landing  position  for  a  different  orientation.  Landing  positions  were 
unaftected  by  the  splotch  of  dots  superimposed  in,  or,  or  just  outside  the  target  form. 
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^  Sketch  of  stimulus  in  the  search  experiment.  Subjects  fixated  the  crosshair  which  could  be  at  one  of  6 
positions  between  the  two  clusters  of  letters.  One  cluster  contained  the  target  letter  T.  The  probablility  of  the  T 
^^learing  on  the  left  was  either  .8  or  ,6,  cued  before  each  trial.  The  task  was  to  report  the  orientation  of  the  T. 
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FIXATION  POSITION  (deg)  FIXATION  POSITION  (deg) 


'C 
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Kguie^’  Latency  of  first  saccade  (left)  and  interval  between  saccades  (right)  for  two  different  probabilities  as  a 
funcbpn  of  initial  fixation  position.  Solid  line;  High  probability  location  on  the  left;  dashed  line:  high  orobabilitv 
location  on  the  right.  » 


